Replication to advance science: changes in ANESTHESIOLOGY.

209 August 2014 “ WHAT’S new with you?” “Take a look at that!” “Well, that’s certainly different!” We are interested in novelty and are wired to notice changes in our environment. Advances in technology mean that many of us are more frequently prodded with new stimuli with each passing day. In this issue of AneSTHeSIology, we highlight something which appears on face value not to be new. A group of investigators replicated a score to predict postoperative pulmonary complications in a different group of patients.1 “How boring!” you might think. Why is this in a journal that focuses on new discoveries? let us briefly describe why this article is important and steps this journal is taking to encourage more articles like it. Judging from the number of this type of publications in the specialty, researchers at least are very interested in predicting postoperative complications. Defining risk factors for postoperative complications is important because these factors can provide clues to guide fundamental science research on mechanisms for these complications. From a clinical perspective, knowing with reasonable precision that a patient is at high risk for major complications allows for a more informed discussion with the patient so that they can appreciate and weigh risks and benefits to the surgical procedure. Certain risk factors are amenable to preoperative treatment or correction, and precautions can be taken in high-risk patients to either start preventive therapy or more intensely monitor for the onset of complications and begin treatment early. Despite the popularity of risk scoring articles and their logical application to clinical practice, predictive models are rarely validated beyond the study from which they were devised and even more rarely meet their full clinical potential. A key problem in the use of predictive scores is their lack of predictive accuracy in future patients and different settings. even with a study of hundreds or thousands of patients, the apparent validity of a predictive model is optimistic when compared with how it will perform in future applications. This is true because the model has been specified (i.e., which predictors?) and estimated (i.e., how much to weight each predictor?) on that particular study group, and the likelihood of similarl high performance, even at the same institution in similar patients, is remote. Statistical approaches such as split-sample, cross-validation, and bootstrapping are all elegant forms of internal validation procedures that can be used to estimate a model’s predictive value in future samples.2 Some form of internal validation procedure is required for publication of novel prediction models in AneSTHeSIology. However, these efforts are only estimates and should be viewed as merely a starting place for the validation process. What is really needed is additional work, preferably by outside researchers across different institutions and settings. In short, a prediction study needs to be replicated. To best understand replicability, consider the various degrees of replication as rungs on a ladder. The first rung of the ladder is reproducibility. The concept of reproducibility refers to the ability to confirm a research finding using the same data set.3 Stated simply, a finding is reproducible if it can be confirmed by a second researcher when given access to the same data as that of previous researcher. Reproducibility is a necessary but not sufficient step for the Replication to Advance Science